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Abstract 

Compressed Sensing (CS) is an emerging field that enables reconstruction of a sparse signal x E M" that 
has only k ^ n non-zero coefficients from a small number m ^ n of linear projections. The projections 
are obtained by multiplying a; by a matrix $ e ][j'"xn — called a CS matrix — where fc < m <C n. 
In this work, we ask the following question: given the triplet {k,m,n} that defines the CS problem 
size, what are the deterministic Umits on the performance of the best CS matrix in We select 

Restricted Isometry as the performance metric. We derive two deterministic converse bounds and one 
deterministic achievable bound on the Restricted Isometry for matrices in K"'^" in terms of n, m and k. 
The first converse bound (structural bound) is derived by exploiting the intricate relationships between 
the singular values of sub-matrices and the complete matrix. The second converse bound (packing bound) 
and the achievable bound (covering bound) are derived by recognizing the equivalence of CS matrices 
to codes on Grassmannian spaces. Simulations reveal that random Gaussian $ provide far from optimal 
performance. The derivation of the three bounds offers several new geometric insights that relate optimal 
CS matrices to equi-angular tight frames, the Welch bound, codes on Grassmannian spaces, and the 
Generalized Pythagorean Theorem (GPT). 
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I. Introduction 

A. Compressed Sensing 

Many signal processing applications focus on identifying and estimating a few significant coefficients 
fi-om a high dimension vector. The wisdom behind this approach is the ubiquitous compressibihty of 
signals: most of the information contained in a signal often resides in just a few large coefficients. Tradi- 
tional sensing, compression and processing systems first acquire the entire data, apply a transformation 
to the data, and then discard most of the coefficients; we retain only a small number of significant 
coefficients. Clearly, it is wasteful to sense and compute on all of the coefficients when most coefficients 
will be discarded at a later stage. This naturally begs the question: can we sense compressible signals 
in a compressible way? In other words, can we sense only that portion of the signal that will not be 
thrown away? The ground-breaking work of compressed sensing (CS) pioneered by Candes et al. [1] and 
Donoho [2] answers this question in the affirmative. 

Candes et al. [1] and Donoho [2] have demonstrated that the information contained in the few 
significant coefficients can be captured (encoded) by a small number of random linear projections. The 
original signal can then be reconstructed (decoded) from these random projections using an appropriate 
decoding scheme. Consider a discrete-time signal a; G that has only k <^ n non-zero coefficients. 
CS posits that it is unnecessary to measure all the n values of x; rather, we can recover x from a 
small number of projections onto an incoherent basis [1,2]. To measure (encode) x, we compute the 
measurement vector y € containing m linear projections of x via the matrix-vector multiplication 
y = <^x, where $ G j^mxn matrix. The CS theory asserts that we can reconstruct (decode) x 

given y and $ using m <^n measurements, provided certain requirements on $ are satisfied. 

It can be shown that if the CS matrix $ is constructed by filUng its m x n entries randomly from an 
i.i.d. Gaussian distribution, then with probability one, m = k + l measurements are sufficient to encode 
X. In particular, with probabihty one, x can be reconstructed exactly from y G W", m>k + \, using Iq 
minimization [3]: 

min ||a;||o subject to $x = y, (1) 

where ||a;||o = {H^Xi : Xj ^ 0}. However, signal recovery algorithms using as few as m = A; -I- 1 
measurements require a search in each of the (^) subspaces that could contain the significant signal 
coefficients. Consequently the complexity of the algorithm to recover x using (1) is NP complete [4]. 
Fortunately, at the expense of acquiring shghtly more measurements, we can recover x from y thru a 
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convex relaxation of (1); the complexity of the resulting recovery algorithm can be made polynomial. 
With m A;log(n/A;) measurements, the solution to the £i minimization (which can be solved with 
cubic complexity) given by 



coincides with the solution of (1) for $ constructed from i.i.d. Gaussian distribution [1,2]. The abihty 
to recover sparse signals easily (in polynomial time) from the small number of CS measurements is one 
of the main reasons why CS has enjoyed tremendous attention in the research community over the last 
few years [5]. 

B. What is a Good Compressed Sensing Matrix? 

The CS matrix $ plays a vital role in both data acquisition and the subsequent recovery of sparse 
signals. Not only do the properties of $ dictate how much information we capture about the signal x, 
but they also determines the ease of reconstructing x from the measurements y. In this paper, we select 
Restricted Isometry as proposed by Candes and Tao [6] as the metric to determine whether a given $ 
is a good candidate for CS data acquisition. The reason we choose this metric is because several key 
results in CS depend on the Restricted Isometry properties of $ [1,6-18]. 

C. Restricted Isometry Property 

Definition 1: For each integer k = 1,2, define the Restricted Isometry constant Sk of a matrix 
$ G ]^"ixn ^Y^Q smallest number such that 



holds for all non-zero vectors x E'MP that satisfy < ||a;||o < A;. 

Note that a "good" CS matrix has a small Restricted Isometry constant Sk. 

As noted before, a number of key results in CS involve the Restricted Isometry constant. We highlight 
two results, both taken from [18]. Theorem 1 shows that if the Restricted Isometry constant S2k of order 
2k is sufficiently small, then CS recovery is guaranteed to be tractable. Theorem 2 shows that the same 
condition on guarantees robustness in CS recovery when the measurements are corrupted with bounded 
additive noise. 

For Theorems 1 and 2, we assume x is any signal in M" and not necessarily /c-sparse. Let Xk € 
be the best A;-term approximation of x, ie, Xk is obtained by taking x and setting all but the k largest 
magnitude entries to zero. Note that if x is fc-sparse, then x = Xk- 



min llxlli subject to ^x = y 



(2) 



{l-dk)\\x\\l<\\^x\\l<{l + 5k)\\x\\l 



(3) 
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Theorem 1: [18, Theorem 1.1, page 2] If 52k satisfies 52k < V^— 1> then the solution x* to the £i 
minimization (2) obeys 

\\x* - x\\e, < Co\\x - XkWe,, (4) 

and 

\\x* - x\\e^ < Cok~^^^\\x - XkWii (5) 

for some constant Co- In particular, if x is fc-sparse, then the recovery is exact. 

Theorem 1 asserts that the solution of the ii minimization is exact when x is fc-sparse and when 
52k is sufficiently small. Rather than solving the intractable £o minimization of (1) directly, we obtain 
the same solution using the tractable ii minimization as given by (2). Note that with only m = k + 1 
measurements, the CS matrix caimot satisfy the condition on 52k- However, with more measurements 
and with specific construction of CS matrices (which can be either deterministic or stochastic), it can be 
shown that the condition in Theorem 1 can be satisfied surely or with high probability [1,2,7,9]. 

The same upper-bound on 52k serves as a sufficient condition for the recovery to be robust in the 
presence of bounded additive noise. Consider noisy measurements 

y = ^x + z, (6) 

where z is an unknown noise vector that is bounded WzWe^ < e. Consider the convex optimization problem 

min \\x\\i^ subject to \\y — ^x\\i^ < e. (7) 

The following Theorem shows that sufficiently small Restricted Isometry constant guarantees 
robust CS recovery of fc-sparse signals in the presence of bounded measurement noise. Specifically, if 
the noise is bounded, then the error in signal recovery is also bounded. 

Theorem 2: [18, Theorem 1.2, page 3] If 52k < — 1 and ||^;||^2 < e, then the solution x* to the 
optimization (7) obeys 

\\x* - xWe^ < CoA;-^/2||x - Xk\U, + Cie (8) 

with the same constant Co as in Theorem 1 and some fixed Ci. In particular, if x is fc-sparse, then 

\\x* — xWe^ < Ci€. 

The central role played by the Restricted Isometry in CS begs the question: what is the best (smallest) 
Restricted Isometry constant we can hope to attain for a CS matrix in R"*^" for a given triplet n, m and 
kl This key question is what we investigate in this paper. 
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D. Notations and Definitions 

We define CS problem size as the triad of numbers {n, m, k}. For a fixed problem size, the CS matrix 
$ is chosen from W^^"-. As we shall see shortly, the Restricted Isometry of $ depends on the singular 
values of the m x fc-sized submatrices of <J> (which are (^*) in number.) While we focus mainly on real 
valued several of the results we derive hold for complex valued $ as well; we will explicitly mention 
if the results are applicable in the complex domain. 

We exploit the fact that the measurements y depend only on k out of the n columns of $ when 
X is fc-sparse; the k columns are the ones that correspond to the indices of the non-zero entries in x. 
As a consequence, the Restricted Isometry constant depends directly on the properties of the m x A; 
sized submatrices of This observation motivates us to consider the singular values of submatrices 
$p G j^mxfc fonned by selecting only k of the n columns of $. The index p € [1, 2, (^*)] uniquely 
identifies the set of columns chosen from the complete matrix <I>. Let the singular values of the matrix <^ 
be Si, S2,..., Sra, where 5*1 > 5*2 > ... > S„i > 0.' Furthermore, let the singular values of the submatrix 
$p be Sp,i, Sp,2,-, Sp^k, where Sp,i > Sp,2 > •■• > Sp^k > 0. 

We define Pmax{^, k) > and Pmm{^, k) > 0, respectively, as the maximum and the minimum of 
the singular values of every submatrix of $ of size m x k, that is, 

Pmax(^, A;) = (max{sp,i})^ and pmini^, k) = {mm{sp k}f . (9) 
p p 

Clearly, for a fc-sparse signal x, we have that 

p,ni,,i<i>,k)\\x\\l < \\^x\\l < p^^^{^,k)\\x\\l, (10) 

where both inequalities in the above equation are tight. We define the Restricted Isometry Property Ratio 
(RIP ratio) as 

This ratio plays the role of the square of the condition number of $ when it is applied exclusively to the 
domain of /c-sparse signals. 

We denote the differential operator ^ by D, i.e., D[f{x)] = ■^f{x), D'^[f{x)] = -^/{x) and so 
on. We use the same variable name x when we consider polynomials such as the characteristic equations 
of $ and its submatrices. Any ambiguity with reference to our A;-sparse signal x is resolved from the 
context in which it is presented. 

'We restrict our attention only to the largest m singular values of $ because the remaining n — m singular values are zero. 
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E. Contributions 

Existing results on Restricted Isometry in the CS literature provide achievability results on specific 
constructions of $ (such as random i.i.d. Gaussian) [1,2,7,9]. Most of the results are stochastic, along 
the lines of: "constructing a matrix according to a prescribed method (such as populating the matrix with 
i.i.d. Gaussian or i.i.d. Bernoulli entries) yields a CS matrix that satisfies RIP with a given dk with high 
probability [7]". There have also been recent results on deterministic achievability, where a deterministic 
construction of $ is proved to satisfy a statistical RIP (more on statistical RIP in Section V) for some 
constant 5k [9]. 

Optimal CS reconstruction is intimately related to Gelfand and Kolmogrov widths of £p balls, an 
area that was extensively studied in the late 1970's and early 1980's by Kashin, Gluskin, and Gamaev 
[19-23]. Connections between the two fields have been recognized by Donoho [2] and others [7,24-26] 
and have led to several deterministic bounds on Restricted Isometry. However, these prior results offer 
little insight into the structure of CS matrices that are optimal in the Restricted Isometry sense. In this 
work, we fill this gap by revealing the intricate relationships between optimal CS matrices and several 
well known results in coding theory and frame theory. As an important consequence of our work, we 
stumble upon a result that extends the well-known Welch bound [29] to higher orders. 

In this paper, we derive two deterministic converse bounds for the RIP ratio based respectively on the 
structural properties of the CS matrix and packing of subspaces in M.'^^'^. Additionally, we also derive 
a deterministic achievable bound for the RIP ratio, which says that we can certainly find a $ G M.'^^"' 
that has a better RIP ratio than our new achievable bound. 

The first bound we derive is a deterministic converse bound on the RIP ratio, called the structural 
bound. It is a function of the problem size {n, m, k}. The key insight that we use to derive it is that the 
singular values {sp,i} of $p (submatrices of $) are severely constrained by the singular values {Sj} of 
the complete matrix For example, the interlacing inequality [27] requires that 

Sp,i < Si for all i = 1, 2, k, 

and therefore the {sp,i}'s caimot take on arbitrary values. We explore the intricate relationships between 
the two sets {sp^i} and {Si} of singular values to derive the structural bound on the RIP ratio. The 
structural bound places a limit on the best (smallest) RIP ratio that is attainable for matrices in R"*^". 
Theorem 4 in Section II is the main result on this bound. The Theorem holds not only for real valued 
but also for complex valued We also show how the structural bound is related to equi-angular tight 
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frames (ETF) [28], Welch bound [29], and the the Generahzed Pythagorean Theorem [30,31]. 

The second bound is also a deterministic converse bound on the RIP, called the packing bound. It is 
applicable to real-valued In this paper, we derive the packing bound for the limited case of = 2; 
however, we plan to extend the results for A; > 2 in a future paper. The need for the packing bound is 
motivated by the fact that the structural bound is loose for large values of n (in comparison to m and 
k); the packing bound offers a tighter bound in the regime of large n. The packing bound is derived 
from an entirely different perspective: we consider the n columns of $ as vectors in R"* and show that 
minimizing the RIP ratio is equivalent to finding n vectors in where the angular separation of every 
pair of vectors is as large as possible. In other words, we show that deriving the optimal RIP ratio for 
A; = 2 is equivalent to optimal packing in Grassmannian spaces. The main result of the packing bound 
is presented in Theorem 5 in Section II. 

The two converse bounds are applicable to every matrix in R"*^". Therefore, the results provide 
bounds on the best (smallest) possible RIP ratio that any CS matrix can achieve for a given problem size. 
We also discuss which of the two converse bounds presented is likely to be the tighter bound, based on 
the particular selection of {k, m, n}. 

The third bound we derive is a deterministic achievable bound, called the covering bound, which 
also exploits the equivalence of CS matrices to packing in a grasmannian space. We use a Theorem 
derived independently by Chabauty, Shannon, and Wyner [32-34] that uses covering arguments in order 
to guarantee achievability. The result is of the following flavor: there exists at least one $ in R"^x" that 
has RIP ratio equal to or better (smaller) than the one given by the covering bound. The main result of 
this bound is Theorem 6 in Section 11. Again, our derivation considers only the case A; = 2; we plan to 
extend the results for A; > 2 in a future publication. 

After deriving the bounds, we present extensive numerical results from which we draw several 
important observations and conclusions. 

A summary of our key results made visual in the Figures is given below: 

1) We see from Figure 1 that there is a large gap between the RIP ratio of Gaussian matrices and the 
achievable bounds. This observation points to the existence of CS matrices that are far superior 
than Gaussian matrices in terms of the RIP ratio. 

^Although we derive the packing bound for real valued $ in this paper, we can use similar arguments to easily derive another 
packing bound that applies for complex valued 
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2) We see from Figure 1 that, for small values of n, the structural bound is the tighter of the two 
converse bounds, whereas for large values of n, the packing bound is tighter. 

3) Figure 1 also presents a graphical illustration of all three bounds and compares the bounds to the 
RIP ratio of Gaussian matrices, as well as the best known matrix for the given problem size. We use 
the results of Conway, Hardin, and Sloane [35-37], who have run extensive computer simulations 
to extract the best known packings in Grassmannian spaces. 

4) While the structural bound for the RIP ratio has been derived for any n, m, and k, we presently 
have the packing and covering bounds only for k = 2. Figure 2 depicts the structural bound for 
A; = 4 and A; = 10. 

5) Figure 3 shows how the parameters of the structural bound can shed Ught on the statistical RIP that 
was proposed recently [8,9,38,39]. In particular, we demonstrate a way to estimate the singular 
values of a randomly chosen submatrix of $ in Section V. 

Furthermore, we show that the structural bound for A; = 2 is equivalent to the Welch bound [29]. The 
structural bound for A; > 3 can be thought of as extensions to the Welch bound to higher orders k > 2. 
We state our extension to the Welch bound exphcitly in Theorem 23. 

The results we present offer lower bounds for the RIP ratio R{^, k) defined in (11). The reason we 
do not derive the bounds directly for the Restricted Isometry constant Sk defined in (3) is because Sk 
changes as we multiply $ by a scalar. On the other hand, the RIP ratio -R($, A;) is invariant to the scaUng 
of We relate the lower bound on A;) to lower bounds on using Theorem 3 below. 

First, fix $ G R"'^'", $ 7^ 0, and select the smallest ei G M and €2 G M such that 

(1 - ei)\\x\\l < W^xWl < (1 + £2)11x11,2^ (12) 

holds for all fc-sparse x. From the definition of the Restricted Isometry constant 6^ in (3), we have 
6k = maxjei, 62}- Now consider a scalar a G M, a / 0, and let = a<&. select the smallest e[ G M and 
€2 G M such that 

(1 - e[)\\x\\l < W^'xWl < (1 + e'2)\\x\\l (13) 

holds for all A;-sparse x. The Restricted Isometry constant for which we denote by is given by 
6'^. = max{e'^, 62}. In this setting, we present the choice of the scalar a that minimizes 5^, i.e., we find 
the CS matrix that has the best Restricted Isometry constant in the family of matrices <!>' = a$. 

Theorem 3: Let $' = a<I>, where a G M, a / is a scalar, and let ei, €2, e'^, €2 be the smallest real 
numbers such that the conditions of (12) and (13) are satisfied for A;-sparse signals x, where x ^ 0. Let 
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Sk = max{ei, €2} and (5^ = m£Lx{e']^, 62}. If 

« = w-^ , (14) 

2 + e2 - ei 

then the following results hold: 

1) Si = e\ = e'2, 

2) 6'^ < 6k. 

Proof: We first note that = a and = a. Using the choice of a given in (14), we express e'l 

and 62 in terms of ei and 62 to obtain 

/ £1 + ^2 J / ei + £2 

ei = — — and €2 



2 + €2 — ei 2 + 62 — ei 

Therefore, e[ = Since S'j^ = max{ei,e2}, we have S'p. = e[ = e'2, proving the first statement of the 
Theorem. 

In order to prove the second statement of the Theorem, we consider three cases. 
Case 1: ei = €2. 

In this case, we have a = 1, and so e[ = ei and e'2 = t2- Therefore b'^ = 6^, which satisfies the second 

statement of the Theorem. 

Case 2: ei < €2- Under this assumption, 6^ = ^2- 

Because ei < €2, we have (2 — ei + €2) > 2, and so for the choice of a in (14), we have 

l + 6'2 = , , / ^ (1 + 62) 

^ + €2 — ei 
< l + e2. 

Therefore, we have <5jfc = < €2 = S^, which verifies the second statement of the Theorem. 
Case 3: ei > e2. Under this assumption, 6^ = t\. 

First we recognize that (1 — ei) > and (1 + €2) > 0, and therefore, the sum (1 — ei) + (1 + £2) = 
(2 - ei + 62) > 0. Since e\ > €2, we have (2 - ei + €2) < 2. Hence, < (2 - ei + 62) < 2. For the 
choice of a in (14), we have 

Z + €2 — ei 
> 1-ei. 

Therefore, we have S'f, = e'^ < ei = Sk. 

Hence, we have S'j^ < Sk for all the 3 cases, proving the second statement of the Theorem. □ 
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Theorem 3 shows how we can pick a in order to obtain the matrix with the best Restricted Isometry 
constant (namely S'^) from the family of matrices a$. Below, we express (5^ in terms of the RIP ratio, 
which is scale invariant. 

Recognizing that 1 — ei = Pmm{^, k) and 1 + €2 = Pmax(^, k), we can write S'l^ as 

6'l. = — — (from Theorem 3) 

2 + 62-61 

(l + e2)-(l-ei) 
(l + e2) + (l-ei) 

Pmin 

{^,k)+p^i^{^,k) 

= ^^i^ (15) 
R{^,k) + l 

Using (15), we infer that the condition 62k < — 1 in Theorems 1 and 2 is equivalent to the 
condition R{^,2k) < + 1. More precisely, if ^ satisfies R{^,2k) < \/2 + 1, then there exists a 
scalar a 7^ such that the scaled matrix satisfies 62k < — 1- 

F. Organization of the Paper 

In Section n, we present the three main Theorems (Theorems 4, 5 and 6) with the three deterministic 
bounds on the RIP ratio. We also present a graphical illustration of the three bounds in comparison with 
RIP ratios of real matrices (including Gaussian matrices) taken from M™^". In Section 111 we prove 
the structural bound for the RIP ratio and present numerous results on the properties of this bound. We 
also offer a tighter structural bound (Theorem 7) when the singular values of <1> are known. We end the 
Section by offering a geometric interpretation of the bounds and their relationship to the Generahzed 
Pythagorean Theorem and to equi-angular tight frames (ETF). In Section IV, we show the equivalence of 
optimizing the RIP ratio of CS matrices and optimal packing in Grassmannian spaces. We use packing 
and covering arguments to prove the deterministic packing bound and the achievable bound. In Section V, 
we demonstrate how one can extend the results on deterministic bounds to statistical-RIP. We conclude 
in Section VI. 
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II. Main Results 

A. Structural Bound: Converse Deterministic Bound for RIP Ratio 

Theorem 4: (Structural Bound) Let $ be an m x n matrix over M or C and < m < n. Let k be 
an integer such that < A; < m and define the fc'th degree polynomial 

fk{x) ^ D^-^ [x"-"^(x - I)'"] . (16) 

The following results are true: 

1) The k zeros of fk{x) are real and he in the interval (0, 1]. 

2) Let > r| > ...r^ be the zeros of fk{x). Then, the following lower bound on the RIP ratio holds 



2 

R{^,k)>^. (17) 



3) Let Si > S2 > ... > Sjn > be the m singular values of Equahty in (17) is achieved if and 
only if all of the following three conditions are satisfied: 

a) Si = S2 = ■■■ = Sm- 

b) The largest singular values of every m x k submatrix of $ are all equal. That is, Sp^i = Sq^i 
for all p.qe {1,2... 

c) The smallest singular values of every mx k submatrix of $ are all equal. That is, Sp^k = Sq^k 
for all p,qe {1,2,..., (l)}. 

The above Theorem asserts that we cannot find a $ in M"*^" that has a better (smaller) RIP ratio 
R{^,k) than rf/r^. 

B. Packing Bound: Converse Deterministic Bound for RIP Ratio for k = 2 
Definition 2: Define the function Cm{(3), f3 G [0,7r] by 

CmiP) = / sin"* ^ ada. 
Jo 

Theorem 5: (Packing Bound) Let 9 G (0, tt) be the solution to the equation 

Cmie)='-^, (18) 

n 

and let qi = cot^ (6). Then there exists no $ G M™x" that satisfies 2) < gi. 

The above Theorem asserts that we cannot find a $ in that has a better (smaller) RIP ratio 

i?($,2) than qi. Note that because Cm{P) increases monotonically with P G (0,7r), the solution to (18) 
can easily be solved numerically. 
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C. Covering Bound: Achievable Deterministic Bound for RIP Ratio for k = 2 

Theorem 6: ( Covering Bound) Let 9 £ (0, vr) be the solution to the equation 



cm{e) = 




and let q2 = cot^ (|). Then, there exists a $ G ]^rnxn g^^,]^ ^^^^ 2) < q2. 

The above Theorem guarantees the existence of a $ in M™^" that has a better (smaller) RIP ratio 
R{^, 2) than 92- 

D. Graphical Illustration of the Bounds for k = 2 

Figure 1 illustrates the three bounds presented above, along with the performance of a Gaussian CS 
matrix and the "best" known CS matrix. The results are given for k = 2, for which we can compute all 
the three bounds presented above. Furthermore, for A; = 2, we show in Section IV that finding the best CS 
matrix (in terms of RIP ratio) is equivalent to a coding problem in Grassmannian spaces. Fortunately, we 
have the best known codes available from the work of Conway, Hardin, and Sloane [35-37]; therefore, 
we can compute the best CS matrices for a given choice of n and m. Hence we can compare the three 
bounds with the best known CS matrix as well as provide comparisons to a randomly generated CS 
matrix using i.i.d. Gaussian distribution. In Figure 1, we consider respectively m = 3, m = 6, m = 8, 
m = 10, m = 12, and m = 16. We vary n along the horizontal axis and plot the RIP ratio and the 
bounds on the vertical axis. 

The following observations can be made from Figure 1. 

1) The Gaussian CS matrix construction has a much higher RIP ratio in comparison to the three 
bounds, revealing the existence of CS matrices in M™^" that will offer much better performance. 
Specifically, there is a substantial gap between the Gaussian RIP ratio and the achievable bound. 

2) The best known CS matrix (obtained from the work of Conway, Hardin and Sloane [35-37]) in 
j^mxn jj^g ^jp lower than the covering bound and greater than the structural and 
packing bounds. Therefore, we can think of the covering bound as an upper bound and the structural 
and packing bounds as lower bounds; together, they define a band within which the RIP ratio of 
the optimal CS matrix resides. Note that the best known $ lies within this band. In fact, the RIP 
ratio curve for the best $ performs significantly better than the achievable bound. 

3) Regarding the converse bound, the structural bound is stronger for smaller values of n, whereas 
the packing bound is stronger for larger values of n. In fact, for n < m{m + l)/2, the best CS 
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Fig. 1. RIP ratio bounds for k — 2 and different values of m and n. We compare the three bounds to the RIP ratio of Gaussian 
matrices as well as the best icnown matrix for the given problem size. The RIP ratio of Gaussian $ shown is the geometric mean of 
the RIP ratio of 10,000 instantiations of randomly generated m x n Gaussian matrices. The RIP ratio of the best known <I> is based 
on the results of Conway, Hardin, and Sloane [35-37] on the best known packings in Grassmannian spaces. 
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matrices are governed by the structural bound, whereas for n > m{m + l)/2, the packing bound 
governs the behavior.^ In Section HI, we see the significance of this transition point in n and relate 
it to equi-angular tight frames. 

III. Structural Bound for the RIP Ratio 

In this Section, we prove Theorem 4 by first presenting a series of results. We also uncover the 
properties of the proposed lower bound on the RIP ratio. 

A. Bound for RIP Ratio when Singular Values of $ are Known 

First, we present a lower bound on the RIP ratio when the singular values of $ are known. 

Theorem 7: Let $ be an m x n matrix over R or C and < m < n. Let Si > S2 > ■■■ > Sm be 
the m singular values of Let the singular values of the submatrix $p be Sp^i > Sp^2 > •■• > for 
allpG{l,2,...,(^)}.Let 

gk{x) = D^-'^ [x"{x - Sf){x - Si)...{x - SD] . (19) 

Let rf >r2> ■■■rf. be the zeros of gk{x). Then, the following results are true: 

1) The k zeros of gkix) are real and lie in the interval {0,Sf]. 

2) /9max(^,/j) > r'l p,nin{^,k) < r^, and > ^. 

k 

3) Equality in all three inequalities above is attained if and only if the following two conditions are 
satisfied: 

a) The largest singular values of every m x k submatrix of $ are all equal. That is, Sp^i = Sq^i 
for all p,qe{l,2,...,{l)}. 

b) The smallest singular values of every mx k submatrix of $ are all equal. That is, Sp^k = Sq^k 
for all p,qe {1,2,..., (l)}. 

The proof of this result hinges on a Theorem of Robert C. Thompson [27] that relates the singular 
values of all submatrices of a given size to the singular values of the complete matrix. We present the 
statement of Thompson's Theorem here for completeness; its proof and other results on singular values 
and eigen values of submatrices are dealt with in [27, 40^7] . 

Theorem 8: [27, Theorem 4] Let A be an m x n matrix with singular values ai > 02 > ... > 
o^min{m,n}- Let Bp be an Z X A; submatrix of A for some fixed I G {1, 2, m} and k G {1, 2, n} with 

^For m = 16, the structural bound seems to govern for higher n as well. 
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singular values Pp^i > j3p^2 > •■• > ^p,rmn{i,kY index p that identifies the submatrix amongst alH x A; 
submatrices of A. Set 

fp{x) = {X- Pli){x- Plmin{l,k})^ 

fix) = {x-al){x-al)...{x-a'^^^^^^^y). 
Then, the following result is true: 

Y^l-min{l,k}f = — D 

^ •'^^ ^ [m-iy. {n-k)\ 

where the summation is taken over all the submatrices of A of size I x k. 

Proof of Theorem 7: The zeros of gk{x) are real and he in the closed interval [0, Sf] as a consequence 
of the Gauss-Lucas Theorem [48]. Recall that the Gauss-Lucas Theorem asserts that every convex set 
in the complex plane containing all the zeros of a polynomial also contains all its critical points (the 
zeros of the derivative of the chosen polynomial). In our setting, we consider the polynomial G{x) = 

x"'~"^{x — Sf){x — S'2)...(.x — S^) that has all its zeros in the closed interval [0, Sf] on the real number 
line. Differentiating the polynomial G{x) {n — k) times and applying the Gauss-Lucas Theorem at each 
step, we infer that the k zeros of gk{x) are real and he in the interval [0,5^]. Finally, we observe that 
X = cannot be a zero of the polynomial gk{x), because x = is a zero of G{x) of order (n — m), 
whereas we differentiate {n — k) > (n — m) times. Thus we have established the first statement of 
Theorem 7.^ 

To prove the second statement of Theorem 7, we apply Theorem 8 (Thompson's Theorem) to our 
setting. Consider the complete matrix $ of size m x n and the collection of m x k sized submatrices 

$p for p=l,2, (l). We obtain 

_l_^n-fc ^^n-m^^ _ ^2^^^ _ ^2)_ _ ^2^^] ^ J^ix - S^.^X - sl,)...{x - S^,). (20) 

The above equation relates the singular value polynomial of <I> to the singular value polynomials of the 
submatrices Recall that the singular value polynomial of a matrix A is the polynomial whose roots 
are the squares of the singular values of the matrix.^ Equation (20) asserts that the sum of the singular 

"* Additionally, note that when n — k > m, x — S'i cannot be a zero of gkix). 

^Equivalently, the singular value polynomial of A is the characteristic polynomial of the Grammian A or AA^ , whichever 
has the higher degree. 



X 



m—k 



X 



-min{m,m} 
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value polynomials of the submatrices $p is a constant multiple of the (n — A;)'th derivative of the singular 
value polynomial of 

Since rf,r|, ...,r^ are the roots of gk{x), we have 

^(^ _ rl){x - rl)...{x - rl) = D^-' [x'^— (x - Sl){x - Sl)...{x - S^)] , (21) 

where the constant n\/k\ is needed to equalize the coefficients of x'^ in the LHS and RHS. From (20) 
and (21), we have 

{x - rl){x - r|)...(x - rl) = J^i^ - ~ 4,2)-i^ " 4,k)- (^2) 

p 

We are now in a position to prove the second statement of Theorem 7. First, we note that pmax(^, k) = 
maxpjsp^} and Pmm{^,k) = mmp{Sp i}. The first inequafity in the second statement implies that 
maxpjsp ^} > . For the sake of a contradiction, assume that §2 1 < rl for all p £ {l,2, ...{I)}. Under 
this assumption, (rf — s^j) > for all p and i and hence each of the polynomial {x — Spi){x — Sp2)---{x — 
Sp iJ is strictly positive when evaluated at x = rf. Consequently, the sum in the RHS of (22) evaluated 
at X = is strictly positive, which is a contradiction, because the LHS of (22) evaluates to zero for 
X = rf. Therefore, we require rf < max{Sp ^} = Pmax{^, k). 

We can show that > maxjspj^.} = Pmin(^,^) using a similar argument as above. Note that we 
need to consider the additional nuance of the sign of (x — ]^)(x — Sp 2)---{x — j.) when evaluating 
at X = rl with the assumption ^ > r|. The sign of (x — ^)(x — 2)---{^ ~ 4 k) either positive 
or negative depending on whether k is even or odd, respectively. Finally, since Pmiix{^,k) > r\ and 
Pmin(^5 k) < rl, we have k) > and hence the second statement of Theorem 7 is established. 

We now prove the third statement of Theorem 7. When s ^ are equal for all p £ {1, 2, (^)}, 
we observe that with x = Sp ^, the RHS of (22) evaluates to zero. Hence s^ ^ must equal rf for some 
i G {1, 2, k}. Since we have established in the second statement of Theorem 7 that we cannot have a 
zero of {x — rf){x — r2)---{x — rl) greater than Sp ^, it follows that rf = Sp i for all p. Similarly, when 
Sp J, are equal for all possible p, we have r| = ^. 

Conversely, suppose r\ = max{sp^} is true. For the sake of a contradiction assume that the values 
of s^i, p G {1,2,..., (^)} are not all equal. In particular, pick pi such that s^^ ^ < maxjsp]^}. Then 
the polynomial corresponding to p = pi within the summation in RHS in (22) is strictly positive when 
evaluated at rf. Consequently, r\ cannot be a zero of (22), yielding a contradiction. A similar argument 
applies to the case when = minjSp ^}. Thus we have established the third statement of Theorem 7, 
and therefore the proof of Theorem 7 is complete. □ 
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While Theorem 7 appUes to the case where the singular values of the CS matrix $ are known, 
Theorem 4 applies to all matrices $ of size m x n. In our quest for deterministic bounds for the RIP 
ratio, Theorem 4 therefore is of central importance. To establish Theorem 4, we explore the following 

question: what choice of 81,82, ■■■,8'^^ gives the most conservative bound for the RIP ratio when we 

^2 

invoke Theorem 7? We show below (Theorem 12) that the ratio is minimized when all the singular 
values of <1> are equal. Therefore this minimum ratio serves as a universal bound for RIP applicable to 
all CS matrices $ of size mxn and leads to the proof of Theorem 4.^ Toward this goal, we investigate 
the nature of the dependence of rf on Si Si..., Si in Section HI-B. 



B. The Zeros of gk{x) and the Singular Values of ^ 

In this Section, we treat each r?, i G {1, 2, k} as a function of m variables Sf, Sl S^ with 



the goal to determine the optimal choice of Sj's that minimize the ratio The exact functional form 



is given in (21). First, we present how an infinitesimal increment in one of the 5|'s affects the values of 
rfs. 

Theorem 9: Let G{x) = — 8D{x - 8l)...{x — 8^) and let rf > > ... > be the zeros 

of the polynomial gk{x) = D''-''[G{x)]. Then, we have 



d{s]) 



evaluated at x 



evaluated at x = rf. 



(23) 



Proof: Taking the partial derivative of (21) with respect to the variable Si while keeping Sl Sl 8] 
fixed, we have 



n! d 



-ix-rl)...ix-rl) 



d 



_jjn-k \^x"{x- 8l)...{x- 8,_ 



(24) 



Mdisl)"- ^' d{Sl)' 

Treating the quantity inside the U^'^W as a product of {x - Sf) and a;"~"^(x - S'|)(a:: - S^)...{x - 5^) 
and using Leibnitz's Theorem [49] for the (n — A;)'th derivative of a product, we obtain 

D^-' [x--"^{x - 8l){x - 8l)...{x - SD] ={x- Sl)D'^-^ [x"— (x - Sl){x - - SD] 

+{n - k)D^-^-^ [x"— (x - Sl){x - Sl)...{x - Sl)] . (25) 



Of course, Theorem 7 provides a tighter bound for the RIP ratio when the singular values of $ are known. 
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Taking partial derivative with respect to Sf (and noting that the second term of RHS in (25) is independent 
of Sj), we can rewrite the RHS of (24) as 

d 



d{Sf) 



= -D^-'' [x"-™(x - Sl){x - Sl)...{x - SD] 



= -D 



n—k 



G{x) 



(26) 



Expanding the LHS of (24), 
n! d 



k\ diSl) 



(x -r^)...{x -r,^) 



ni x ^ 



i=l 



n (^-^') 
i</<fc,/^« 




(27) 



Evaluating (27) atx = rf and noting that only one term in the summation in the RHS of (27) is non-zero, 
we obtain 



n! d 



kl d{S( 



-{x-rf)...{x-rl) 



ni 



l<l<k,l^i 



n\ [n 



KKk 



{x - rf) 



d{Sf 



D"-'^ [G{x)] d{rf] 



X — rr 



d{St] 



(28) 



From (24), (26), and (28), we infer that 



d{s]) 



D^-''G{x) 

5 



evaluated at x 



(29) 



Finally, we use the well known result that if a function f{x) has a zero at x = a of multiplicity 1 (i.e., 

a simple zero), then f{x)/{x — a) evaluated at a: = a is equal to D [f{x)\ evaluated at x = a. Applying 



this result, we have that ^G{x)/{x — rf) evaluated at 



x = r. 



is equal to '^+^G(x) evaluated at 



X = rf, since rf is a zero of ^G{x). Using this in (29), we deduce that 



Ml 

d{S^) 



D 



n-k ( Oix) 



J)n-k+lQ(^x) 



evaluated at x = rf, 



(30) 



which proves the first part of Theorem 9. 
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and, subtracting (37) from (36), we obtain 



x-Sl 
x-Sl 



+ ...+ 



^ _ Q2 



(31) 



(32) 



To prove the second part, denote S{x) = {x — SD{x — S2)---{x — S^), and consider the quantity 

£)"-'=+i[a;G(a;)]. We have 

D^-k+i^xG{x)] = D''-''[D{x''-"'+^S{x)}] 

= L>"-*^ [(n - m + + x"-"^+^L>[S(x)]] 

= (n - m + l)L>"-'= + D''-'' [xD[S{x)]] . 

Alternately, we can use Leibnitz Theorem to expand yielding 

= {n-k + l)^'^-*^ [G{x)] + xD^'-^^^ [^"-'"^(x)] . 

From (31) and (32), we have 

{n-m + l)D''-^[x'^-"'S{x)] + L*""^ [xL>[5(x)]] 

= {n-k + l)D''-^[G{x)\ + xD''-^+'^[x"S{x)] 

and therefore 

L*"-'^ [x"-"*+^L>[5(x)]] = (m - A;)L>"-'= + xL>"-'=+MG(x)] . 

Therefore, 

= ^ {l''^-'^ - mG{x)\ + feL'"-*^ . (35) 

Since D[S{x)\ = S{x) + ... + 5^}, we have 

x"^'D[Six)] = + - + 

Rewriting mD^~^G{x) as 



(33) 



(34) 



(36) 



(37) 



L>"-^ [x"-"^+^L>[5(x)]] - mD''-^G{x) 



= D 



n—k 



x^'-'^Six) 



SI 

-sf 



+ ...+ 



(38) 
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Plugging the above result in (35) yields 



D'^-^+^[x"S{x)\ = -kD''-^ [G{x)] 



X 

+ 



^{P?""-'(Sf)}- 

Substituting (39) in (30) and evaluating at x = rf (noting that D'^~^ [G{x)\ is zero at x = rf), we obtain 
the result in (23) which completes the proof of Theorem 9. □ 
Remark 1: From (23), we have 

which we recognize as Euler's condition for homogeneity of rf on {5f , 5|, S^} of degree 1 [49]. 
Remark 1 implies that when we write as a function of {Sj,S2,---, S^}, we have 

rfiaSf,aSl...,aSl) = arUslSl...,Sl), (41) 

for all a G M+. This result is in agreement with (21), because the homogeneity result can be derived 
from (21) by making a change of variable from x to ax. 
Theorem 10: Under the assumptions of Theorem 9, 



> 0. 



Theorem 10 is significant, because it indicates that increasing for any j can only increase the 
corresponding r|'s. This fact can be exploited if our objective is to maximize the singular values of 
the submatrices of In Donoho's paper [2], the condition CSl requires the smallest singular values 
of the submatrices of $ to exceed a positive constant. The above Theorem indicates the relationship of 
CSl condition to the singular values of the complete matrix <I>. Also, in any practical system, we have 
a bound on the maximum S'j's that can be used, reminiscent of coding with power constraints [50]. In 
such a scenario. Theorem 10 indicates that the best choice for the S^'s is when they are all equal to the 
maximum allowable bound. In order to prove Theorem 10, we require a result on interlacing polynomials. 

Definition 3: Two non-constant polynomials p{x) and q{x) with real coefficients have weakly inter- 
lacing zeros if: 

• their degrees are equal or differ by one, 

• their zeros are all real, and 
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• there exists an ordering such that 



ai < A < "2 < /32 < - <a„</3^< ... 



(42) 



where ai, 0:2, ... are the zeros of one polynomial and /3i,/32, ■. are the zeros of the other. 

If, in the ordering of (42), no equality sign occurs, then p{x) and q{x) have strictly interlacing zeros. 
We use the following result of Hermite and Kakeya [48] to prove Theorem 10. 

Theorem 11: (Hermite-Kakeya) Let p{x) and q{x) be non-constant polynomials in x with real co- 
efficients. Then, p{x) and q{x) have strictly interlacing zeros if and only if, for all /x, A G R such that 
X"^ + fi^ > 0, the polynomial g{x) = \p{x) + fiq{x) has simple, real zeros. 

Proof of Theorem 10: We demonstrate in this proof that the numerator and denominator polynomials 
in the RHS of (23) have the same sign when evaluated at a; = rf. Consider the three polynomials 



and P2{x) are of degree {k — 1), while q{x) is of degree k. We assert that 

1) pi(x) and q{x) have strictly interlacing zeros, and 

2) P2{x) and q{x) have strictly interlacing zeros. 

The first statement above is a straightforward consequence of the interlacing property of a polynomial 
with real zeros and its derivative [48]; we note that pi{x) = D[q{x)]. The second statement follows from 
a direct application of Hermite-Kakeya' s Theorem to P2{x) and q{x). Consequently, for a given zero of 
q{x), say r?, there are an equal number of zeros of pi{x) and P2{x) to the left of r? on the real number 
line. Similarly, there are an equal number of zeros of pi{x) and P2{x) to the right of rf on the real 
number line. Lastly, note that the leading coefficients of all three polynomials (i.e., the coefficient of 
x'^ for q(x) and the coefficients of x^^^ for pi{x) and P2{x)) are positive. Therefore, pi{x) and P2ix) 
have the same sign when evaluated at a; = rf, and hence the RHS of (23) is positive, completing the 
proof. □ 

Finally, we present the main Theorem in this Section, which shows that the choice of S/s that 
minimize the ratio rf/r"^ is when the Sj's are all equal. 

Theorem 12: Under the assumptions of Theorem 9, the ratio rf/r^ is minimized when S\ = S2 = 



pi{x) = L>"-'=+i[G(x)], p2{x) = D 
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Proof: In order to minimize rf/r^, we consider its partial derivatives with respect to the Sj's. At the 
optimal location, we require the partial derivatives to be zero, i.e., 

2\ 1 




1 dirf) 1 d{r 



rjdiSf) rldiS^) 




1 9(r2) _ 1 d{rl) 



(43) 



for all the S/s. We show that the above condition is satisfied with the choice Sf = S2 = ■■■ = S^. 



r{ d{S]) ri d{Sf) 

show that the above com 

Assume that the 5|'s are all equal and non-zero, and denote their common value by 5^. Because of 

d(r^) 

symmetry, the quantities are independent of the choice of j, and we denote the said quantity by 
^T^- Consequently, the terms in the summation of LHS in the Euler homogeneity equation (40) are all 
equal, and hence 

9(52) - 

Therefore, ^ is independent of the index i and hence the condition (43) is satisfied. It remains to 
show that the optimal point just derived is a minimum. A rigorous analysis to prove minimahty involves 
the computation of the Hessian matrix [49] of t- with respect to the Sf's. However, the analysis quickly 
tums intractable. Instead, we fix = 1 and note that the only choice of the S'I's for j = 2, 3, A; that 
satisfy the condition (43) is when they are all equal to unity. Thus, we can check the maxima or minima 
criteria by comparing the value of ^ at 5^ = 1 to another choice for the set {S?}. We pick the set 
Sf = 1, 5*2 = 0, 5*3 = 0, S*^ = for the purpose of comparison, and we immediately see that = 00 
for this choice, because r| = 0. Therefore, the optimal point we have determined is a minimum. □ 
We remark that the above proof reveals that the ratio r?^ / r?^ of any two roots r? and rf^ is minimized 
for a given ii and 12 such that ii < 12. Finally, we note that Theorem 7 together with Theorem 12 proves 
Theorem 4, which is the main result of this Section. 



C. Properties of the Structural Bound 

In this Section, we study the relationships between the structural bound given by Theorem 4 and the 
parameters n, m, and k of Compressed Sensing. Clearly, the properties of the structural bound are tied 
to the properties of the polynomial fk{x) as defined in Theorem 4. We begin by first expressing fk{x) 
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in the standard polynomial form, by carrying out the (n — A;)'th order differentiation in (16). We obtain 

= (44) 

and 

9k{x) = iZi-^)'-' (E S!SI..SQ i^^±j^x^-. (45) 

We ask if the form of fk{x) is similar to one of classic polynomials in the literature [48]. The ratio 
of successive terms of the polynomial suggests that fk{x) is a Gauss Hypergeometric function of the 
second kind: 

_Cj_ ^ [ {k-j-i) {j+iy. ^ 

{j-k){j + n-k + l) 



{j + m-k + l){j + 1 



-X. 



Therefore, 



fk{x) = c -2 Fi {—k,n — k + l]m — k + 1; x) , 



for some constant c. There is very little known about the location of the zeros of hyper-geometric functions 
of the kind described above [51]. However, algorithms to compute the zeros have been recently studied 
[52]. 
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To see the dependence of the structural bound on n, m and k, we plot rf/r^ as a function of n in 
Figure 2. We assume that the Sj's are all equal. Several properties of the bound can be inferred from 
the plots. In particular, 

1) For a given m and k, the ratio rf/r^ increases when we increment n. 

2) For a given m and k, the ratio rf/r'^ approaches a constant as we let n — > oo. In other words, 
rf I r\ can be upper bounded by a constant that is independent of n. 

3) For a given n and m, the ratio r\ / increases when we increment k. 

4) For a given n and k, the ratio rf/r| decreases when we increment m. 

Each of the statements above is stated and proved in the form of Theorems below. Alongside each 
Theorem, a result of the same flavor is proved for the RIP ratio for #, if such a result exists. 

The following Theorem shows that when we keep m and k fixed, the structural bound on the RIP 
ratio is an increasing function of n. 

Theorem 13: Let > r| > ... > r| be the zeros of the polynomial L'"^'"[.t"^™(x — 1)™], and let 
tl>tl> ... > tl be the zeros of the polynomial D(ri+i)-k^^{n+i)-m(^^ _ i)m]_ ^hen, 

d<d 

Proof: We have 

= D"-*^ - 1)"*-^ {(n + 1 - m){x - 1) + mx}] 



= {n + l)D 



n—k 



x"(x - IT-^ [x-[l- 



m 



(46) 



n + 1 

Comparing (46) and (16) reveals that the polynomial fk{x) evaluated for n + 1 and m is of the same 
form (up to a constant) of gk{x) evaluated for n and k with all singular values equal except one with 
value S"^ = ^1 — • Because the singular values are not all equal. Theorem 12 ensures that < p-. 
This completes the proof of Theorem 13. □ 

Theorem 14: Let <I>2 be an m x (n + 1) sized matrix over M or C, and let be an m x n submatrix 
of $2- Then, the RIP ratios of the two matrices satisfy R{(^i,k) < R{^2,k). 

Proof: Let Ai be the set of all squared singular values {s^j} of all submatrices of $i, and let A2 be 
the set of all squared singular values of all submatrices of $2- Since every m x k submatrix of $1 is 
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also a submatrix of we have Ai C A2. Therefore, m£ix{Ai} < m£ix{A2} and min{Ai} > min{A2}. 
Because k) = max{Ai}/ min{Ai} and R{^2, k) = max{A2}/ min{A2}, the result of Theorem 14 

follows as a consequence. □ 
The following Theorem shows that the structural bound can itself be bounded by a quantity that is 
independent of n. 

Theorem 15: Let > r| > ... > be the zeros of the polynomial D'^~^[x^~'^{x — l)"*]. Then, 

rl ^ {mkY{m — k)' 



r| ml 



Proof: We establish the result by deriving an upper bound for rf and a lower bound for r^. Applying 
Viete's Theorem to the polynomial in (44), we have 

rl + rl + ...+rl = —. 

Since each term in the LHS of the above equation is positive, we have 

ri< , (48) 

n 



giving us an upper bound on r\ 

''k 

involving the product rfr2...r^: 



To derive a lower bound on rf,, we begin by using Viete's Theorem for the constant term of (44), 

2 _ \k) 



Since rf > r| for all i < k, v/e have the inequality 



„2/ 2\fe-l ^ \k) 



Applying (48) to the above inequality, we have 

,2 (!^X~\rUr!r^>^^. 



Therefore, we have 



n 



m\ k-1 
^2 > Kk) n 
^k ^ 



m!(n — k)ln^~^ 
(m — fc)!n!(mfe)^~-^ 

ml {n — k)ln^ 

n{m — k)l{mk)^^^ n\ 

> ^^1 , (49) 

n(m — k)l{mk)'' ^ ' 
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where the last inequality (49) is a consequence of > n\/{n — k)l. Combining the inequaUties (48) 
and (49) by taking the ratio (noting that both inequalities have positive LHS and RHS), we obtain the 
inequality in (47), completing the proof of Theorem 15. □ 

Although Theorem 15 gives an upper bound for the structural bound rf/r^ that is independent of n, 
we cannot bound the RIP ratio R{^,k) as we increase n. This is because R{^,k) necessarily increases 
with n. Consequently, as we increase n, the structural bound becomes more loose (this can be seen 
in Figure 1). The packing bound, discussed below in Section IV, provides a tighter bound for large n 
because it captures the growth of -R($, k) with increasing n. 

Theorem 16: Let > > ... > rl be the zeros of the polynomial D'^''^[x^~^{x — 1)"*], and let 
tl>tl> ... > tl be the zeros of the polynomial D"-'=[a;"-"^+^(x - l)'""^]. Then, 

r]_ tl 

Proof: Reducing m by one is equivalent to the following operation: set = 0. The proof of the above 
Theorem follows therefore from a straightforward application of Theorem 12. □ 
Theorem 17: Let $2 be an (m+ 1) x n sized matrix over M or C, and let $1 be an m x n submatrix 
of ^2- Then, the RIP ratios of the two matrices satisfy R{^i,k) < R{^2,k). 

Proof: Let p G {1, 2, (^)} be the index that identifies the set of k columns that are selected from 
the complete matrix to form a submatrix with k columns. Then, ($i)p is a submatrix of ($2)^, and 
consequently, the maximum (minimum) singular value of ($i)p is smaller (greater) than the maximum 
(minimum) singular value of (^•2)^ by the interlacing Theorem for matrices [27]. Thus Theorem 17 is 
established. □ 
Theorem 18: Let > r| > ... > r| be the zeros of the polynomial D'^~'^[x"^~'^{x - 1)™], and let 
tl>tl> ... > tl^^ be the zeros of the polynomial L>"-'=-i[x"-"'+i(x - l)™"!]. Then, 

„2 4.2 



^2 — J.2 
'k ^k+l 

Proof: Since L>'^-'=-i[x"-™+^(x-l)'^-i] = D - 1)'"]] , the roots of the two polynomials 

weakly interlace, as a consequence of the interlacing theorem for a polynomial and its derivative [48]. 
Therefore, r\ < t\ and > t^j^-^ and the proof of Theorem 18 follows as a consequence. □ 
Finally, we state a similar Theorem for matrices. 
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Theorem 19: Let $ be an m x n matrix over M or C. Then, the RIP ratios satisfy -R($, k) < 
R{^,k + 1). 

The proof of Theorem 19 is identical to the proof of Theorem 17. 

D. Geometric Interpretation 

Recall the geometric interpretation of the SVD: The matrix $ with SVD $ = USV'^ can be 
represented as a hyperellipse of dimension m embedded in R". The axes of the hyperelhpse are aligned 
with the column vectors of V with the length of each semi-axis equal to the corresponding singular value. 
Denote this hyperellipse by E{^). It is well known [53] that ||$x||2 is equal to the magnitude of the 
projection of the vector x onto the hyperellipse £'($). Furthermore, the column vectors of U describe 
the orientation of the hyperelhpse in W^, which is the image of $x of the unit sphere in M". That is, 
for ||x||2 = 1 we have 




where the g-wise summation in (50) is the sum of all the terms that are obtained by multiplying q unique 
singular values.^ Equation (50) relates the dimensions of E{<^) to the dimensions of the collection of 
/c-dimensional hyperelhpses E{^p) corresponding to each $p. Note that each of the hyperellipses E{^p) 
lie in a A;-dimensional subspace spanned by k canonical basis vectors. A particularly interesting case is 
when m = k = q, for which (50) reduces to 

{SiS2...Skf = 5^(Sp,lSp,2...Sp,fe)'. (51) 

p 

The above result form = k = qis equivalent to the Generalized Pythagorean Theorem (GPT) [30, 
31]. The GPT states that the square of the fe- volume of a A;-dimensional parallelepiped embedded in an 
n-dimensional Euclidean space is equal to the sum of the squares of the /c-volumes of the (^) projections 
of the parallelepiped on to the distinct /c-dimensional subspaces spanned by the canonical basis vectors. 
Equation (51) implies that the statement of the GPT can be directly carried over from parallelepipeds to 
hyperellipses. 

^For example, consider q = 2, m = 4 and A; = 3 for illustration. Then, '^g_y/isQiSiS2-.Sq)^ = (Sift)^ + (SiSs)^ + 
{8184)'^ + {8283)'^ + {8284)^ + {8384)'^, and X) vvise(sp,iSp,2--Sp,q)^ = (sp,iSp,2)^ + (sp.iSp.s)^ + (sp,2Sp,3)^- 
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Equation (50) extends GPT to arbitrary values of m, k and q.^ The form of the equation motivates 
us to define the g- volume of an ellipse of intrinsic dimension that is greater than q as follows. 

Definition 4: Consider a hyperellipse H of dimension d > q with semi-axes ai, 02, a^. The q- 
volume of H, denoted by Yo\q{H), is defined as 



y q-wise 

Equation (50) therefore relates the g-volumes of E{^) to the g-volumes of E{^p) in the following 
manner: the square of the g-volume of E{^) is proportional to the sum of the squares of the g-volumes 
of E{^p). 

E. Structural Bound for k = 2 

In this subsection, we study the structural bound for the specific case of A; = 2. The motivation for 
studying this case are many fold. First, /c = 2 is the smallest non-trivial case to investigate the RIP ratio. 
For the case k = 1, any matrix $ that has equi-normed columns satisfies R{^, 1) = 1, and therefore 
the structural bound is trivially 1 for any n and m. Secondly, the roots of the polynomial (44) can be 
explicitly evaluated, providing an avenue for analysis. Lastly and most importantly, designing good CS 
matrices for A; = 2 can be shown to be equivalent to well-known problems in coding theory. 

For A; = 2, the form of (44) and (45) reduce to 

n(n — 1) 



J2s]]{n-l)x+[ Yl SlSl\=0 (52) 



and 



n(n — 1) o , m(m — l) „ 

-5^ '-x^ - (ra - l)mx + — ^ ^ = 0. (53) 

2 ^ ^ 2 

We focus on (53), because we are interested in universal bounds for the RIP ratio. The roots of (53) can 

be computed as 

99 ml / mfn — m) 
ri,ri = —± ' ^ ' 



n n V n — 1 



and the structural bound is thus 



1 1 / n—m 



1 - 



m(n— 1) 

'*Note that the hyperellipses are the projections of onto the canonical fc-subspaces only when m = k. 
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Note that as n — >■ oo, the above equation reduces to 



hm 4 = 5^ = ^- (54) 

n — >-oo To 1 / 1 1 

^ 1 — \ — m 



Recall that we derived an upper bound (47) on ^ that is apphcable for any m, k and as n — > oo. 

Substituting A; = 2 in (47), we obtain 

r\ ^ (2mf{m-2)\ _ Am? _ 4 ^^^^ 
r1 ~ m\ m(m — 1) 1 — - 

Comparison of (54) and (55) reveals that (54) offers a tighter bound on r'l/r'j, than (55). 

We now make some interesting connections between good CS matrices for A; = 2 and coding theory. 

The key result that provides the segue is Theorem 20. 

Theorem 20: Let A=[a\ 02 03 ... a„] be an m x n matrix over M or C comprising n>2 columns 

ai, 02, ...an of size m x 1, with ||ai II2 > 0. Construct the m x n matrix B as 

ai 02 03 On 



B 



(56) 



|ai||2 ||a2||2 \\0-Z\\2 \\0-n\\2_ 

obtained by scahng every column of A independently so that the ^2 norm of every column of B is unity. 

Then, the RIP ratios of ^ and S satisfy R{B, 2) < R{A, 2). 

Proof: We prove the Theorem by considering two cases: n = 2 and n > 2. 

Case 1: n = 2. 

Since A has only two columns, the only submatrix of A of size m x 2 is ^ itself. Therefore, the RIP 
ratio is simply the ratio of the square of the two singular values of A. Let the I2 norms of column vectors 
ai and 02 be ||ai||2 = di and ||a2||2 = d2. Let the angular distance between ai and 02 be 6, given by 
cos(^) = I (ai • 02) |/(c?ic?2), where | {a\ ■ 02) | is the absolute value of the dot product of a\ and 02. 

Let S\ and 51 be the squared singular values of A, with S\> 82- Since the squared singular values 
of A are the eigen values of the grammian matrix A^ A, we compute A^ A as 

A-A=\ 1. (57) 



did2Cos{9) d? 



2 

2\2 



Thus Sf and S2 are the zeros of the characteristic polynomial of (57) given by x — {df + 6^2) + 
(o?id2)^ sin^(^). Computing the roots of this polynomial yields 



dl + dl + J {dl + di) - 4 {did2f sm\e) 
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and 



Si 



dl + dl 



The RIP ratio of A is given by 



R{A,2) 



Si _ dl + dl + ^ {d\ + di)' - 4 (dids)' sin2(^) 



SI 



where 



d:i + dl-^J{dl + dlf -A{did2f sin2 {6) 
1 + ^Jl - us{n\e) 

1 - - vs\.-a\ey 

(2did2) 



(58) 



is the ratio of the geometric mean and the arithmetic mean of d\ and d^. From (58), we infer that in 
order to minimize R{A, 2) for a fixed 6, we require u to be as large as possible. Since v is the ratio 
of geometric mean and arithmetic mean of d\ and d'^, maximum value of v is attained when di = d2, 



yielding 



1. 



Since the angular separation of the column vectors remains invariant while constructing B from A 
using (56), and we have equal column norms in B, we infer that R{B,2) < R{A,2) from the above 
arguments. Thus we have proved Theorem 20 for the case n = 2. 

Note that when di = ^2 = d, we have 



Sf = d^il + cos(^)) , 



Sl = d^{l- cos{d)) , 



and 



Sl _ 1 + cos(g) 
Sl ~ 1 - cos{9) 



(59) 
(60) 

(61) 



While Case 1 is apphcable to matrices with two columns, we extend the result to include any mxn 
matrix in Case 2. 
Case 2: n > 2. 

Consider the following three m x 2 sized submatrices Ap^, Ap^, and Ap^ of A, where Po,Pi,P2 are 
indices from 1, 2, (2): 

1) Ap^ is obtained by selecting the two columns of A that have the minimum angular distance among 
all pairs of column vectors of A. Let the singular values of Ap^ be Sp^^i and Sp^^2 with Sp^^i > Spo,2- 
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2) Ap^ is the m X 2 submatrix of A whose largest singular value Sp^^i is the maximum of all singular 
values of submatrices of A of size m x 2. In other words, 

= argmax{sp,i}, p = 1, 2, (2) . 

Let the singular values of Ap^ be Sp^^i and Sp^^2 with Sp^^i > Spi,2- 

3) Ap^ is the m x 2 submatrix of A whose smallest singular value Sp2,2 is the minimum of all singular 
values of submatrices of A of size m x 2. In other words, 

P2 = argmm{sp,2}, p = 1, 2, (3) • 

Let the singular values of Ap^ be Sp^,! and Sp2,2 with Sp^^i > Sp2,2- 
We also consider an m x 2 submatrix Bp^ of B, using the same index po defined above. The angular 
separation between column vectors are invariant to scaling of columns; therefore Bp^ contains the two 
columns of B that have the minimum angular distance among all pairs of column vectors of B. We 
denote the singular values of Bp^ as ti and t2 with ti>t2. 
From the definition of RIP ratio, we have 

R(A,2) = ^. (62) 



*P2,2 



Since Sp^^i > Sp^^i and 5^3,2 < Spo,2> we have 



„2 - 2 
*P2,2 So,2 



Invoking the result we have established from Case 1 for the m x 2 sized matrices Ap^ and Bp^, we have 
R{Ap^,2) > R{Bp^,2) and so 

> i. (64) 



„2 - f2- 



Let us consider the RIP ratio of the matrix B. Since every column of B has equal norm, we assert that 
the RIP ratio of B is govemed completely by the submatrix Bp^. To see this, we first note that when 
the column norms are equal, the condition number of an m x 2 matrix is dictated only by the angular 
separation of its two constituent column vectors. Specifically, if the two columns of an m x 2 matrix 
have equal £2 norm of d and their angular separation is 9, then the squared singular values are given 
by (59) and (60). Furthermore, the squared condition number is given by (61). The squared condition 
number is a monotonically decreasing function of 9 in the range < 9 < ir/2. Therefore, the RIP ratio 
of B is given by 

i?(i?,2) = |. (65) 
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Using the results of (62), (63), (64) and (65) and cascading the inequaUties, we obtain R{A, 2) > R{B, 2), 
and so the proof of Theorem 20 is complete. □ 

The above Theorem has important consequences for the RIP ratio of order k = 2. 

Remark 2: Theorem 20 reveals that among the set of all matrices that can be obtained by scaling 
each column of a given matrix independently, the RIP ratio for = 2 is minimized when the column 
norms are all equal. 

We use the above results to study the properties of a CS matrix $ that attains the structural bound 
for A; = 2. First, Theorem 4 (third statement) requires that $p has the same pair of squared singular 
values for all p. Second, we deduce from Theorem 20 that the columns of $ are all equi-normed. If the 
contrary were true, then equalizing the column norms will yield a matrix with smaller RIP ratio, which 
violates Theorem 4. Since this result is of importance, we state it in the form of a Theorem. 

Theorem 21: If an m x n matrix $ over R or C satisfies (17) with equaUty, then $ has equi-normed 
columns. 

Furthermore, we show that $ that satisfies (17) with equality is an equi-angular tight frame (ETF) [28]. 
From [28, 29], we list the three conditions for a matrix A to be an ETF: 

1) The columns of A are unit normed, 

2) The absolute values of the dot product of every pair of columns of A are same, i.e., the columns 
are equi-angular. 

3) AA* = {n/m)Imxm- 

Theorem 22: Let an m x n matrix <I> over M or C satisfy (17) with equality, and let ^ be scaled 
such that its m squared singular values are equal to n/m each, i.e., 5^ = 51 = ■■■S^ = n/m. Then, $ 
is an ETF. 

Proof: Let $ = USV* be the singular value decomposition for We check the three conditions for ETF, 
starting with the third condition. We have AA* = USV*VSU* = US'^U* = {n/m)UIU* = {n/m)I, 
satisfying the third condition. The second condition is satisfied because every pair of columns have the 
same set of singular values. If the angle between two equi-normed columns is 9 and the common norm is 
d, then the squared singular values of the m x 2 sized submatrix comprising only of the two said columns 
are given by (59) and (60). To verify the first condition, we note that the norms of each column of $ 
are the same, say (f. It remains to show that this norm is unity. Substituting Sl = S2 = ■■■S^ = n/m 
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in (52), we see that sf and S2 are the roots of the quadratic equation 



n n(m — 1) 

x^-2x + ^7 -{ = 0. (66) 

m[n — 1) 

Therefore the sum of the roots of the quadratic equation is given by 

sl + sl = 2. (67) 

From (59) and (60), we have 

sj + sl = 2(f, (68) 

and hence we infer that = 1 by comparing (67) and (68). Thus $ satisfies all three conditions for an 
ETF. □ 
The relationship between the structural bound and ETFs can be used to make statements about the 
set of allowable pairs {m, n) that meet the structural bound. Results from [28] reveal that an ETF of size 
m X n exists only when n < ^m{m + 1) for real ETFs and n < for complex ETFs. In addition, n 
and m should satisfy strict integer constraints. 

F. Structural Bound as Extension of the Welch Bound 

Definition 5: The coherence of a matrix A, denoted by /x(^) is defined as the largest absolute iimer 
product between any two columns Oj, aj of A. That is, for A with n columns, 

fi{A)= max ,1 ■ (69) 

Recall the classical result of Welch that states that the coherence of a m x n matrix in R or C is always 



greater than or equal to y rn(n^i) • ^^^^ known that $ is an ETF if and only if the coherence of $ 
satisfies the Welch bound with equality [28,29]. As a consequence of Theorem 22, we infer that if $ 
satisfies (17) with equality, then the coherence of ^ meets the Welch bound. 

We make the claim that the structural bound for > 2 extends the Welch bound from pairs of 
column vectors of a matrix to fe- tuples of column vectors. We use the least singular value of a submatrix 
as a way to extend the notion of coherence to a A;-tuple of vectors. Specifically, we wish to maximize 
the (normahzed) least singular value of the submatrices in order to keep the submatrices as far apart as 
possible, in some sense.^ However, Theorem 7 asserts that the least singular value has an upper bound. 
Based on this insight, we extend the Welch bound explicitly in the following Theorem. 

'without normalizing the least singular value, it can be increased arbitrarily by simply scaUng the matrix. As we shall see, 
the normalization is done based on the largest singular value of 3>. 
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Theorem 23: (Extension of Welch Bound) Let $ be an m x n matrix over R or C with < m < n. 
Let k be an integer such that < A; < m, and let fk{x) be the fc'th degree polynomial given by 

fkix) = [a;"-™(x - 1)™]. Let rf > > ... > rl be the zeros of fk{x). Let Si be the largest 

singular value of Let Sp i > Sp 2 > ••• > Sp,fc, be the k singular values of the m x k sized submatrix 
$p of <I>, where p G {1, 2, (^)} is the index that identifies the submatrix. Then the smallest Sp^fe, after 
normahzation by Sf, is bounded by 

I '^/''' ]<rl (70) 

Proof: Let -Si > 5*2 > ... > Sm > be the m singular values of Define two more polynomials gk{x) 
and hk{x), both of degree k, given by 

gk{,x) = D^-' [x"{x - Sl){x - Sl)...{x - 5^)] and = D^''' [x"{x - Sfr] . 

Let the zeros of gk{x) be > t| > ... > t^, and the zeros of hk{x) be «f > u| > ... > u^.. 
Based on the result of Theorem 7, we have 

min{4 J < tl (71) 

From Theorem 10, we see that tj < for all i = 1,2, ...,k, because we can think of the polynomial 
hk{x) as being obtained by increasing each of 82,8^, to S^. Theorem 10 guarantees that the zeros 
of hk{x) are greater than the corresponding zeros of gk{x). Therefore, < u\ and so 

mm{s2 .^} < 4. (72) 

While (71) is a tighter bound than (72), the latter equation has the advantage that it depends only on S\ 
and holds for all values of Sf < Sf, for i = 2, 3, m. In the final step of the proof, we use the result 
uf = Sfrf which follows by noting that hk{x) is obtained by making the change of variable x — > Six 
in fk{x). Substituting in (72), we obtain minpjSp^} < Slr^, and the proof is complete. □ 
Based on the proof of Theorem 7, we infer that (70) holds with equality if and only if Sp^k = Slr^. 
for all p e {1, 2, (^)}. This extends the notion of ETFs, which for k = 2, require only the angular 
separation between every pair of column vectors to be the same. Note that a necessary condition for the 
equality of (70) to hold is that Sf = S2 = ■.■ = S^. Recently, Datta, Howard and Cochran have also 
proposed extensions to the Welch bound [54]. 
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IV. Packing and Covering Bounds for the RIP Ratio 

The motivation to derive another bound for the RIP ratio comes from the perspective that we can 
view the CS matrix as a collection of column vectors in M™. We need to spread these vectors as far 
away from each other as possible in order to ensure that the singular values of its fc-column submatrices 
have good condition numbers. Increasing the number of columns (i.e., increasing n) leads to crowding 
of these vectors in R"* that leads to a deterioration of the RIP ratio. We make these notions precise for 
k = 2. For A; > 2, the exact nature of the packing bounds are yet elusive. 

As we saw from Theorem 20, we need restrict our attention only to matrices of equi-normed columns. 
Therefore, the problem of designing good CS matrices for A; = 2 is equivalent to finding arrangements 
of n lines in such that the minimum angle between the pairs of lines is maximized. This problem 
has been studied extensively by Conway, Hardin, and Sloane [35]. Furthermore, converse and achievable 
bounds can been derived by studying a related problem, namely of arrangements of 2n points on a 
Euclidean sphere in M™. The latter problem has been studied independently by Chabauty, Shannon, and 
Wyner [32-34]. We state the main results that are relevant to our problem of bounding the RIP ratio. 
For a detailed description and derivation of the relevant results from coding theory, see [55]. 

Definition 6: The area of a spherical cap of radius ^ on an m dimensional Euclidean sphere of unit 
length is given by 

CmiP) = km sin'" 2 ada, 
Jo 

where km is given by 

_ 27r("^-i)/2 

Note that 2Cm(7r) gives the surface area of a unit sphere in W^. Packing the surface of the m-dimensional 
sphere using spherical caps gives rise to an upper bound on the minimum angle 6 that can be attained 
between the pairs of 2n points on the sphere. The upper bound f^max is given by Cm (^^) = '^"^^^^ ■ 
Consequently, we obtain a lower bound on the RIP ratio, which is captured in Theorem 5 as the packing 
bound. 

Proof of Theorem 5: Using (61) to relate the minimum angular separation 6 and the RIP ratio R{^, 2), 
we obtain 

«(*,2) = i±=ffi = cot^(^). (73, 

1 — cos(6') \2/ 

The statement follows from the fact that for any arrangement of 2n points on the surface of the unit 
sphere in W^, the minimum angle between pairs of points is less than 9 defined above, as a result of 
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packing. Note that the form of Theorem 5 is obtained by making a change of variable 6/2^6. □ 
Furthermore, covering arguments can be used to make a statement on achievabihty. Based on the 
results of Chabauty, Shannon, and Wyner [32-34, 55, 56] we can guarantee the existence of an arrangement 
of 2n points on the Euclidean sphere in M™ where the angular distance between every pair of points is 
at least as large as 9, where Cm{0) = '""^^^^ ■ Theorem 6 captures the achievable bound on the RIP ratio 
obtained using the above arguments. 

While we have successfully derived the structural bound for any n, m and k, the derivation of packing 
and covering bounds for A; > 2 remains an open problem. The main challenge is that Theorem 20 cannot 
be extended beyond = 2. In fact, it is easy to construct matrices where the RIP ratio for A; > 2 increases 
when we equalize the column norms. 

V. Relevance of the Structural Bounds in Statistical RIP 

It can be argued that the definition of RIP is too restrictive in the sense that the RIP constant 5k 
and the RIP ratio R{^, k) depend on extremal values of the singular values of the submatrices. Because 
the number (^) of submatrices of $ is astronomical, it is very unlikely that a randomly selected k- 
sparse signal has the exact sparsity pattern corresponding to the submatrix of $ with extreme values for 
the singular values. Rather than requiring every m x /c-sized submatrix of $ have their singular values 
bounded, we can conceive of a statistical RIP where we allow a small fraction of the submatrices to 
have singular values outside of the bounds. Along these lines, Tropp [38, 39], Calderbank, Howard, and 
Jafarpour [9] and Gurevich and Hadani [8] have proposed the notion of Statistical RIP. 

We demonstrate how the parameters of the structural bound play an important role in capturing the 
estimates of singular values in a randomly chosen m x fe-sized submatrix of The motivation comes 
from the fact that the quantities and (of Theorems 4 and 7) in some sense capture the mean of the 
squared singular values ^ and ^, respectively. In fact, while proving Theorems 4 and 7, we have 
shown that if the s^ys are not all equal, then some of the s^ys he to the left of the real number hne 
from rf and some lie to the right of rf. Similarly for the set of ^ and r^. This leads one to wonder if in 
fact rf is in some sense, the mean of ^. In other words, is it possible that the zeros of the polynomial 
in (19) are in fact estimates of the squared singular values of a randomly selected submatrix of 

We ran the following simulation to test our intuition. We picked a single $ of moderate size mx n 
with a prescribed set of singular values (all ones) and randomly selected a large number of submatrices 
of $ of size m X k. We computed the k singular values of each of the selected submatrices. We plot 
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0.1 0.2 0.3 0.4 

Squared singular value 

(a) n = 100, m = 14, fc = 12 



0.2 0.3 0.4 
Squared singular value 



(b) n = 100, m = 20, fc = 12 




0.2 0.3 0.4 0.5 0.6 
Squared singular value 



0.4 0.6 O.i 

Squared singular value 



(c) n = 100, m = 30, it = 12 (d) n = 100, m = 60, fc = 12 

Fig. 3. Histograms of the squared singular values of submatrices of In each of the plots, we fix n, m and k. We then plot, in 
the same figure, the histograms for the 1st, Ath, 8th and 12th squared singular values Sp i, Sp_4, Sp g, and Sp 12 of 25, 000 randomly 
chosen m x k sized submatrices of $. The complete matrix $ is chosen with m prescribed singular values all equal to unity. The 
four red vertical lines in each plot correspond to the values of the 1st, Ath, Sth and 12th zeros of the polynomial in (19). Note the 
excellent match between the histogram peaks and the polynomial zeros; this observation suggests that the zeros of the polynomial 
in (19) are in fact estimators (a mean, in some sense) of the squared singular values of randomly chosen m x k submatrices of $. 



the k histograms of the respective squared singular values and compare them against the r?'s. Figure 3 
shows these plots for a set of 4 out of k singular values (we chose only 4 in order to prevent clutter in 
the plot). Note that the rf's provide remarkably good estimates for the s^^'s. This observation strongly 
suggests that the roots of the polynomial in (19) play a crucial role in statistical RIP, irrespective of 
which of the two converse bounds (structural or packing) is the tighter deterministic bound. We believe 
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that this observation serves as a starting point for analysis in statistical RIP. 



VI. Conclusions 

In this paper, we have derived two deterministic converse bounds for RIP ratio. The first bound is 
based on structural bounds for singular values of submatrices and the second bound is based on packing 
arguments. We have also derived a deterministic achievable bound on RIP ratio using covering arguments. 
The derivation of the three bounds offer rich geometric interpretation and illuminate the relationships 
between CS matrices and equi-angular tight frames, codes on Grassmannian spaces and Euclidean spheres, 
and the GeneraUzed Pythagorean Theorem. 

A summary of our key results is given below: 

1) There is a large gap between the RIP ratio of Gaussian matrices and the achievable bounds. This 
observation points to the existence of CS matrices that are far superior than Gaussian matrices in 
terms of the RIP ratio. 

2) For small values of n, the structural bound is the tighter of the two converse bounds, whereas for 
large values of n, the packing bound is tighter. 

3) We compared the three bounds to the RIP ratio of Gaussian matrices, as well as the best known 
matrix for the given problem size. We used the results of Conway, Hardin, and Sloane [35-37], 
who have run extensive computer simulations to extract the best known packings in Grassmannian 
spaces. 

4) While the structural bound for the RIP ratio has been derived for any n, m, and k, we presently have 
the packing and covering bounds only for k = 2. We believe that the result for k = 2 establishes 
a starting point to investigate the packing and covering bounds for k > 2. 

5) The parameters of the structural bound can shed light on the statistical RIP that was proposed 
recently [8,9]. In particular, we demonstrated a way to estimate the singular values of a randomly 
chosen submatrix of 

6) We showed that the structural bound for A; = 2 is equivalent to the Welch bound [29]. We have 
used the structural bound for A; > 3 to extend the Welch bound to higher orders. 

The present study of deterministic bounds for RIP opens up many interesting research questions. 
While we have derived deterministic RIP ratio bounds that apply to all matrices in M"*^", it would be 
valuable to derive the deterministic RIP ratio bounds for special class of CS matrices within M'"^", 
such as {—1,1} matrices, sparse matrices, and matrices that have block zeroes that appear in Distributed 
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Compressed Sensing [3]. Analysis of these special class of matrices would also help measure the penalty in 
terms of the increase in RIP ratio we need to tolerate. Next, we plan to characterize the exact relationship 
between the stochastic RIP described in Section V and the parameters of the structural bound. Finally, 
packing and covering bounds for > 2 remains an open problem. 
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